Continual Learning through Retrieval and Imagination

Continual learning is an intellectual ability of artificial agents to learn new streaming labels from sequential data. The main impediment continual catastrophic forgetting, a severe performance degradation on previously learned tasks. Although simply replaying all previous data or continuously adding the model parameters could alleviate issue, it impractical in real-world applications due limited available resources. Inspired by mechanism human brain deepen its past impression, we propose novel framework, Deep Retrieval and Imagination (DRI), which consists two components: 1) embedding network that constructs unified space without arrival tasks; 2) generative produce additional (imaginary) based memory. By retrieving experiences corresponding imaginary data, DRI distills knowledge rebalances further mitigate forgetting. Theoretical analysis demonstrates can reduce loss approximation error improve robustness through retrieval imagination, bringing better generalizability network. Extensive experiments show performs significantly than existing state-of-the-art methods effectively alleviates